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ABSTRACT 

This study addresses the need to develop research-based 
criteria for science teacher educators to use in preparing teachers to 
critically evaluate and select web-based resources for their students* use. 
The study focuses on the cognitive load imposed on the learner for tasks 
required in using text, illustrations, and other features of multi- media, 
hyper-linked web-resources. This study seeks to use terminology that is 
meaningful and clear to teachers to develop criteria for the evaluation of 
the learners* cognitive load in the use of web sites. These criteria will be 
useful for science teacher educators to prepare teachers to critically 
evaluate web resources. Drawing from prior research, this study formulates 
ways to assess the cognitive load of instructional plans available on the 
web. Preservice teachers in an elementary and secondary science methods class 
were asked to use the criteria to assess pre-selected science WebQuests, a 
type of teacher- male inquiry instruction compatible with science process 
learning and the learning cycle. Results from individual preservice teacher *s 
ratings using the criteria, and from follow-up in-class discussions are 
presented and used to identify criteria understood and found useful to 
elementary and secondary preservice teachers, and those criteria needing 
further refinement. (Author) 
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Abstract 

This study addresses the need to develop research-based criteria for science teacher 
educators to use in preparing teachers to critically evaluate and select web-based 
resources for their students’ use. The study focuses on the cognitive load imposed on the 
learner for tasks required in using text, illustrations, and other features of multi-media, 
hyper-linked web-resources. We seek to use terminology that is meaningful and clear to 
teachers to develop criteria for the evaluation of the learners' cognitive load in the use of 
web sites. These criteria will be useful for science teacher educators to prepare teachers 
to critically evaluate web resources. Drawing from their prior research, the authors 
formulate ways to assess the cognitive load of instructional plans available on the web. 
Preservice teachers in an elementary and a secondary science methods class were asked 
to use the criteria to assess pre-selected science WebQuests, a type of teacher-made 
inquiry instruction compatible with science process learning and the learning cycle. 
Results from individual preservice teacher’s ratings using the criteria, and from follow-up 
in-class discussions are presented and used to identify criteria understood and found 
useful to elementary and secondary preservice teachers, and those criteria needing further 
refinement. 
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Subject/Problem 

This study addresses the need to develop research-based criteria for science 
teacher educators to use in preparing teachers to critically evaluate the appropriateness of 
web-based science resources for their elementary, middle school, and high school 
learners. In earlier research, we compared teachers’ and scientists’ perceptions of the 
credibility of printed and electronic information sources. As we had expected, scientists 
were more critical about many of the information sources than were teachers, especially 
elementary teachers. We also found interesting differences between elementary and 
secondary teachers, suggesting in part, that teachers’ views are shaped by the 
developmental needs of the students that they teach (Klemm, Iding & Speitel, 2001). 

We then worked with high school biology students to have them develop their 
own criteria for critically evaluating websites generally and the scientific information in 
them specifically. The students were oriented to basic website terminology and given 
three general areas of website evaluation to consider: validity, credibility, and 
presentation (based on work from Farah, 1995; Rader, 1998; and adapted by Nguyen, 
2000). The students indicated the process of learning to critically evaluate website as 
most valuable to them, and they generated more extensive lists of criteria than generated 
at the outset of the intervention (Iding, Landsman & Nguyen, in press). In other research, 
we compared self-ratings of preservice educational psychology students with those of 
computer science students on two aspects of critical evaluation of websites. Interestingly, 
the education students rated themselves lower than the computer science students on 
evaluating websites, but higher than the computer science in competence on evaluating 
course-related topics (Iding, Crosby, Auemheimer & Klemm, 2002). Pertinent to the 
present research, these findings suggested to us the need to further examine teachers’ 
views in critically evaluating websites. 

In another study (Iding & Klemm, 2002), we examined teachers’ determinations 
of appropriate cognitive load associated with websites and other multimedia material. We 
defined cognitive load in accord with Chandler and Sweller (1991) and Sweller and 
Chander (1994), referring to the extent to which cognitive resources are taken up by tasks 
that are either relevant to or unrelated to learning. For example, “split attention effects” 
occur when learners must coordinate separate text and illustrations (Sweller & Chandler, 
1994). Kischner (2002), Bannert (2002), and Valcke (2002) further differentiate types of 
cognitive load. Specifically, we were interested in how preservice teachers 
conceptualized the notion of cognitive load in critiquing and selecting web-based 
materials for their students, which they were free to select. When asked to develop their 
own criteria for assessing cognitive load as an aspect of selecting a web site for their 
students to use, teachers included “developmentally/cognitively appropriate” most often, 
then information load, and also criteria pertaining to “visual density” (too much/not 
enough visuals). Among our findings was the need to develop consensus among 
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educators as to the criteria, and the terms or language used in them, together with 
examples for the students. 

Iding (2000) reviewed some of the relevant effects emerging from this line of 
research and related research that would be relevant to development of criteria. In that 
article, she summarized research as follows: 

“Some aspects of textual and multimedia design that can reduce cognitive load 
on learners include: Presenting smaller, more manageable chunks of texts-with- 
illustrations (Mayer, 1999); Presenting illustrations and relevant texts in 
proximity to minimize the visual search process (Chandler & Sweller, 1991; 
Sweller & Chandler, 1994); Minimizing extraneous information or sensory 
input (Mayer, 1999); and Using auditory input to accompany diagrams, to 
minimize excessive resources that may be spent in visual search or construction 
of visual representation while at the same time reading accompanying text (a 
second and possibly competing visual process) (Mayer, 1999)” (p. 410). 

Design/Procedure 

The present study builds on our previous work, this time looking at the cognitive 
load factors used by in critically evaluating websites for their students’ use in preselected 
web-based inquiry lessons. Participants were preservice enrolled in an elementary 
science (N=21) or secondary science (N=19) methods course. Participation was 
voluntary and anonymous, and took place during 2 class sessions. 

For this study, we used an existing web-based inquiry instructional model, the 
WebQuest, as a vehicle to support our research on what teachers need to learn and be 
able to do in order to critically evaluate and select websites for their students to use. A 
WebQuest is “an inquiry oriented activity in which most or all of the information used by 
learners is drawn from the Web” with the focus on using information, not searching for it. 
According to WebQuest developers Dodge and March (1995), WebQuests support 
learners’ thinking at the levels of analysis, synthesis, and evaluation 
(http://webquest.sdsu.edu/overview.htm). A WebQuest is an example of an assigned 
search of web site links. Each WebQuest contains a resource list of web sites that were 
selected as appropriate and relevant by the teacher(s) who developed the lesson. 
Moreover, the WebQuest site provides an introductory lesson orienting teachers to the 
model, called A WebQuest About WebQuests, with elementary and middle school/high 
school versions available at the WebQuest home page 
(http://edweb.sdsu.edu/webquest/webquest.html.) 

We modified the introductory WebQuest lesson worksheet. First, we changed the 
web resource list so that it contained only science web sites, rather than a sampling of 
different subject areas. Importantly, for this study, we included one science website in 
common for both the preservice groups. Second, we added to the introductory 
worksheet, brief instruction about website evaluation and cognitive load, plus criteria 
derived from the recommendations of Iding (2000) and other prior research (discussed 
earlier). We asked participants to use these criteria to critically evaluate each web site. 
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Otherwise, we followed the introductory lesson strategy in the A WebQuest About 
WebQuests. 

The elementary or middle school/high school preservice teachers were assigned 
specific WebQuest science lessons to examine. Participants recorded responses to each 
worksheet/questionnaire item as they performed their assigned tasks. Working in groups 
of four, each took on one of the following assigned roles: efficiency expert (examining 
use of time); affiliator (need for collaboration, discussion and consensus); atitudinist 
(examining higher order thinking) or technophile (making the best use of technology). 
Working individually, participants spent about 10 minutes to examine each of the sites 
from the assigned perspective, and to record observations, in keeping with the original 
worksheet. Unlike the original worksheet, we also asked each participant to critically 
evaluate each web site using the criteria we provided. We asked them to evaluate 
cognitive load associated with the web sites using criteria derived from previous 
researchm for example, Iding & Klemm (2002), derived from Farah, (1995 and Rader 
(1998), and adapted in the work of Nguyen (2000), and Iding, Landsman and Nguyen 
(2001), also Iding & Klemm (2002) adapted from Changler & Sweller, 1991 and Sweller 
& Chandler (1994). Then, group members conferred to discuss their individual ratings of 
web sites using our modified worksheet, which emphasized cognitive load criteria. They 
recorded notes during their discussion which aimed at developing a compromise 
consensus in ranking the web sites. As with the original introductory worksheet, we did 
not expect them to reach unanimous agreement. One person from each group then 
reported the group’s thoughts to the class. 

Data Analysis and Findings 

We first ascertained the expertise levels with respect to using web sites. More 
elementary teachers rated themselves as novices with respect to searching for materials 
on the web, selecting materials for their own use as students, selecting materials for their 
use as a teacher, and selecting materials for their students to use (i..e., elementary 
teachers selected the novice category 33 times, whereas secondary teachers selected it 20 
times.) Secondary teachers rated selected the category “expert, could teach others” more 
often (6 times) than did elementary teachers, who selected this category only twice. The 
vast majority of responses from both groups indicated that they consider themselves as 
proficient (selected 44 times for secondary teachers and 46 times for elementary 
teachers.) 

Students were asked to rate each portion of one selected, common WebQuest site 
in terms of information density (1 = too little, 2 = about right amount, 3 = too much) and 
complexity in navigation (1 = easy, 2 = about right, 3 = difficult). Frequencies were 
tabulated in each category. Preliminary examination of the results indicated that the 
largest differences between elementary and secondary teachers’ views of information 
density of the web site were in their ratings of the amount of visual information (m=l.l 
for elementary and m= 1.79 for secondary) and textual information (m = 2.76 for 
elementary and 2.26 for secondary teachers). This indicates that elementary teachers 
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tended to rate the overall information for the particular web site as more dense and the 
textual information as more difficult. 

Students also rated the complexity in navigation of each portion of the selected 
WebQuest. The largest differences between the two groups were found in their ratings for 
introduction (m = 1.81 for elementary; m = 1.00 for secondary) and process or procedure 
(m = 2.38 for elementary; 1.63 for secondary) portions of the web site. The majority of 
elementary teachers rated the introduction section as about right, whereas the majority of 
secondary teachers rated it as easy. More elementary teachers found the process 
(procedures in carrying out the WebQuest tasks) as difficult than did the secondary 
teachers. 

The authors worked together to perform a qualitative analysis of comments that 
students wrote in response to rating one common web site’s information density and 
complexity in navigation. We developed categories that emerged from the data and 
calculated frequencies for comments within categories. To briefly describe these derived 
categories, aspects of cognitive load included brevity, appropriateness of load or 
overload; positive or negative aspects of linking; difficulty or ease of internal navigation; 
presence of choices, usability; time; and interactivity. Elementary teachers had more 
comments, more complete comments and more specific observations than did the 
secondary teachers, as for example, commenting on the color and font size of the web 
site. That elementary teachers had more detailed comments may not be surprising, 
considering that they had been trained in teaching of reading and have had more 
extensive background in working with special education students than did the secondary 
teachers. 



Contribution & Interest to NARST 

This study uses a web-based inquiry instructional model that is now a part of 
science teacher education, the WebQuest, as a vehicle to further understanding of criteria 
teachers need for critical evaluation and selection of web-based resources for science 
teaching. That web-based inquiry lessons are now part of science teacher preparation is 
seen in Abruscato’s introductory discovery methods book for elementary and middle 
grades (2000), where he explains that science WebQuests provide “real reasons” to use 
computers to investigate real world contexts involving science. Abruscato describes 
WebQuests as teacher-prepared discovery projects for students that involves giving 
learners practice in use of science process skills (e.g. observing, classifying, predicting; 
use integrated science process skills (e.g. interpreting data, formulating hypotheses); and 
support of the approach to lesson planning known as the Learning Cycle. 

However, to our knowledge, there is little systematic research to inform teacher 
educators, who now have the need to prepare teachers to critically evaluate and select 
web-based resources for their students to use in web-based inquiry learning. We believe 
that our work links research on cognitive load with research on the need to prepare 
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teachers to critically evaluate and select web sites for student use in web-based inquiry 
lessons and projects. This is a useful way to prepare preservice teachers to make effective 
determinations regarding the selection and incorporation of web-based science materials 
in their classroom instruction. In addition, our present study extends our prior work on 
developing criteria for teachers to use in evaluating the credibility and cognitive load 
aspects of web sites ,and in selecting web-based resources for developmentally 
appropriate instruction. 
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